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Abstract: Failure can be central to faculty research; however, failure produces a vehicle for 
learning. Through an interdisciplinary faculty community, the authors supported each other in 
facing, learning from, and overcoming ‘failed” aspects of research projects. This article reports 
obstacles encountered in conducting Scholarship of Teaching and Learning research and the 
role of a faculty learning community in overcoming these challenges. Research pitfcdls included 
lack of student participants, nonsignificant findings, expectations for understanding related 
course content, technology issues, use of deception, determining the research question, and 
managing bias. Ultimately, the faculty learning community engendered a foundation for 
successful research projects by shared inquiry into these research “failures. ” 
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Researchers understand that unanticipated difficulty is part of the process in human-based 
studies, such as interrupted interviews or technology issues. Indeed, most researchers will agree 
that a well-designed project will result in useful outcomes, even if unexpected obstacles can 
prevent optimum data collection. Clearly, outright “failure” is a result that researchers try to 
avoid. But if we accept that all research is an imperfect and uneven process, then “failures” ought 
to be treated as part of the research landscape, pointing us to different considerations of our 
research design, context, and data. As the adage goes, we learn from our failures, but there exists 
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very little scholarship within The Scholarship on Teaching and Learning that interrogates failure 
as central to a faculty’s research process and, ultimately, publication. Indeed, as instructors we 
may apply failure-as-learning philosophies to teaching, but if we take learning as a lifelong 
endeavor, ought we not consider “failure” in our own work (see Tawfick & Jonaseen, 2013)? 
Ultimately, what does it mean to embrace obstacles and failures as part of our work as 
researchers and to treat it as central to our scholarship? 

Through an interdisciplinary lens, this article addresses how research “failure” can be a 
remarkable learning experience for new and experienced faculty within the context of Faculty 
Learning Communities (FLC). FLCs have been shown to function as a rich, generative space for 
breaking down disciplinary silos and placing faculty who are, as McMahon-Klosterman (2015) 
notes, “relatively knowledgeable and relatively ignorant” (p. 92). By positioning faculty as 
learners, the FLC can function to break apart competitive posturing—something that faculty may 
need to unlearn—and foster learning from the commonality of not knowing. Indeed, it may be in 
the very moment of conflict or the incoherence between disciplinary silos that can call attention 
to alternative research possibilities. The authors of this article contend that failed research can be 
productive because it places us in the space of the unknown, where we are compelled to draw on 
other perspectives, knowledge, and experts that can help us progress our own learning 
experiences, even as we attempt to study students’ learning outcomes. As Halberstam (2011) 
noted, the logic of success can obscure our ability to see the potential in “the unexpected, the 
improvised, and the surprising” (p. 16). 

Indeed, if “failed” research results can tell us something useful about teaching and 
learning, then “failed” or “failure” may not be the best terms to use. However, we start with this 
terminology and premise because one significant argument we make is that this failure can be a 
very real and felt experience for researchers, causing some researchers to abandon their projects. 
In highlighting this experience as exigence, we also want to offer principles from action research 
as an approach that can be helpful in addressing research failures. Action research, as Bradbury 
and Reason (2003) defined, is a “ family of approaches and practices” that emphasizes “doing 
research with, rather than on people” (p. 156, emphasis in original). While Bradbury and Reason 
(2003) directed this principle of collaboration to the researched object/population (p. 157), this 
FLC applied the collaborative principle between researchers, highlighting the ways that cross- 
disciplinary sharing can lead to what Sagor (2000) describes as research that helps improve or 
refine the “actor’s” action. In this case, sharing and collaborating on our different research 
projects, from start to finish, helped us improve and refine not only our research, but it provided 
us opportunities to improve our own teaching as well. 

This article will draw on multiple faculty and disciplinary perspectives to focus on how 
researchers addressed different kinds of “failures” in their projects and to highlight failures as the 
central premise of learning even for seasoned researchers. The projects mentioned in this article 
emerged during the 2014-15 school year through the support from a Faculty Learning 
Community based on the Scholarship on Learning and Teaching, and led by Beth Dietz and 
Catherine Bishop-Clark. The authors are from the various disciplines: English, Communication, 
Nursing, Chemistry, Criminal Justice, and Political Science. 
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Seeking New Directions in Failure 

Link Dich is an assistant professor at Miami University. Dich teaches Asian-American 
Literature, digital writing, and technical writing. Dich’s research focuses on difference and 
identity formation, public spheres, and language practices in digital contexts. Dich has two 
works in press. “Community Enclaves and Public Imaginaries ” (Dich, 2016) is an article that 
examines Asian American identity formation through community and public writing. Her other 
essay, “The Impact of Asian and Asian American Scholarship as a Productive, Contested Site, ” 
(Dich, 2016) argues that conflicting, contested historical accounts can engender a space of 
continuing disciplinary and organizational growth. 

What happens when potential participants are not willing to participate in your project? 
One of the most difficult outcomes of any research project is the lack of participation, especially 
after careful researching, designing, and planning. In many respects, such outcomes mean 
scrapping the whole project and starting over again. 

As an instructor who teaches online courses regularly, it was important to me to 
understand if students learn how to develop writing strategies from my videos, ultimately 
improving their writing on major assignments and their grade for the course. Therefore, I 
designed a project that would test for this learning outcome. With the rapid increase of online 
classes in higher education, I found problematic the assumptions that more technology equates to 
better learning, especially given the existing teaching obligations instructors are negotiating on a 
regular basis (see Moran, 2003). As Porter (2002) argues, we know that technology matters to 
writing, but how and why is less clear. Yet, if there is evidence that video feedback can help 
students learn and write more critically and effectively, teachers may feel more invested in 
incorporating such approaches in their courses, whether online or offline. At the very least, I 
wanted to have data that can help instructors better comprehend students’ perspectives on video 
forms of feedback to writing. 

For data collection, I had hoped to receive student responses from two online courses 
(about 40 students). I asked participants to complete a short survey and for permission to use 
their range of writing submissions in order to observe any changes in the quality of students’ 
writing before and after video feedback. But after multiple emails and reminders sent by a 
colleague, I received zero responses from students for participation. Pushing the restart button 
was the first reaction I had to what I considered a “failed” project. As with many researchers, I 
was immediately filled with self-doubt about my research design: Did I do enough to appeal to 
participants? Should I have asked students to participate earlier in the semester? Did I miss 
something in my project design? It is a rather devastating fact to accept that not even one student 
was willing to take a short survey and provide anonymous permission for their writing. Granted, 
lack of participation is a likelihood for any participant-based research, but one that is not often 
discussed in scholarship. 

These were the issues and questions that I brought to my FLC as I attempted to salvage 
my project. With the help of the FLC, reflecting on this “failed” research allowed me to think 
about issues that may be more immediate to courses than video feedback. In comparing this 
study to my previous research success with student participants in traditional classroom settings, 

I begin to think about how an online course may impact the ways students engage with non- 
obligatory activities. In a traditional classroom, the parameters of time and bodies impact how 
students may feel about completing something like an “optional” survey. Surrounded by peers 
and with the teacher’s gaze, are students more compelled to submit responses to a study? This 
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“failed” project also prompted me to ask what sort of obligations may develop for students’ 
online classes that may be different than traditional ones. If students experience different 
obligations in their online courses than their traditional ones, what are these obligations and how 
do obligations function to impact learning outcomes? What does this mean about the design of 
educational “spaces”? Such questions were prompted by a faculty colleague who was also 
struggling with her own project, particularly with how to remain ethical in regards to research 
deception and disclosure (see “Truth, Lies, and Murphy’s Law”). That is, a fellow FLC 
member’s project provided me a different lens through which to reconsider my project not as a 
failure but as one that may require a different methodology given the digital context of the 
research “site.” Ultimately, I am suggesting that the lack of participants raised other important 
questions about teaching that my colleagues and I may have never thought about examining for 
teaching and research. 

What to do When You Have Non-Significant Findings? 

Jeff Kuznekoff’s teaching and research background is in communication. His primary focus is 
how new communication technology impacts the communication process; however, one of his 
research interests is the use of technology in the classroom. He has two published research 
articles (Kuznekoff & Titsworth, 2013; Kuznekoff, Munz, & Titsworth, 2015) examining student 
texting in class, and how this behavior impacts learning and note taking. The SoTL research 
Kuznekoff describes is adapted from these past studies; however, this particular study ran into 
problems during the analysis phase. 

One of my lines of research involves studying note-taking behaviors of students and the 
impact that technology has on this important student behavior. For example, one of my past 
studies (Kuznekoff & Titsworth, 2013) examined the impact of student texting in class and how 
using a mobile device impacts student learning and note-taking. We found that students who 
abstained from texting (i.e., did not use their mobile device during a simulated lecture) scored 
roughly a letter grade and a half (13% points) higher on a multiple-choice test of lecture material 
than those students actively engaged in responding to messages. A follow-up to that study 
(Kuznekoff et al., 2015) added in the notion of message relevance and message creation to the 
mix. Our more recent study found that students who were responding to messages related to the 
simulated lecture scored roughly on par with the control group, yet students sending or receiving 
messages unrelated to lecture content continued to demonstrate diminished learning. In both 
studies, we used an experimental design to compare several groups against a control group. 

More recently, I modified the design, which has been found to be effective in two 
different studies, and compared student learning and note-taking between students taking notes 
on an iPad and students taking notes by hand. Although I had a comparable number of students 
in each group, when compared to the past studies, I was not able to find statistically significant 
results and the study suffered from relatively lower power. Not only did the study fail to find 
statistical significance, but even if a statistically significant effect were present, I would have 
been unable to detect it with the power level I achieved. 

Part of the issue likely has to do with a particular student skill level, present across all 
groups, and that skill level is note-taking. In college, we typically expect and assume that 
students have already mastered, or are at least experienced with, taking notes in class. However, 
other studies have generally found that students are typically bad at taking notes and miss up to 
40% of important information (Kiewra, 1985; Titsworth & Kiewra, 2004). In this case, it very 
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well could be that the effect I was looking for, ability to take notes and how this might be 
influenced by technology, was not well developed in the control group or even in the 
experimental group. If that is the case, it would make sense that I would be unable to detect a 
difference or even have enough power to detect results. If students are already poor note takers, it 
really would not matter what group or manipulation they were assigned to in my study, their 
notes would all be of relatively low quality. 

In the case of the present study, I could have increased my sample size; however, after 
doing a power analysis to determine the sample size needed to achieve acceptable power (.80), I 
would need several hundred participants, which I would not realistically be able to achieve at 
present. Although this research project ended in a dead end (i.e., not finding statistical 
significance and being unable to detect differences if they were there), I did have an interesting 
finding. It very well could be that the skill I was looking to compare is so poorly developed that I 
won’t be able to find differences between groups of students. Perhaps what would be more 
interesting is examining the current note-taking skill level of students and developing educational 
interventions that help students develop effective note-taking skill. Ample evidence indicates that 
note-taking is a key skill that can aid student learning (Kiewra et al., 1991; Rickards & 

Friedman, 1978); however, students may not realize how bad at note-taking they are. In addition, 
it is also worth noting my own assumptions, and likely the assumptions of other faculty 
members, regarding student note-taking ability. I assumed that college students are already 
skilled note takers and that assumption impacted the design of this study; however, one plausible 
explanation is that my assumption is inaccurate and actually studying that assumption might be a 
fruitful area of research. Although I did not find statistically significant results in my study, I do 
think the finding was an interesting one and opened up an opportunity and invitation to future 
research. 

So ... What is a Chiral Carbon Anyway? 

Janet Marshall teaches organic chemistry and a variety of courses related to the chemistry of 
food and cooking, including a study abroad course in Italy. Her laboratory-based research 
focuses on experiment development, and two of her recent publications, co-authored with 
undergraduate students, describe adding an element of discovery to the organic chemistry lab 
(Marshall & Comminos, 2013; Marshall & Hodge,2016). Her interest in SoTL prompted her to 
join this multi-disciplinary faculty learning community (FLC) to explore pedagogical-based 
research. 

Over the past several years. I’ve developed and refined iPad exercises for my 
introductory and organic chemistry courses. In these exercises, students use the technology to 
better understand topics such as molecular geometry, shape, and stereochemistry. Images of 
molecules, such as melatonin and nicotine, are shown in 3-dimensional space and can be easily 
rotated using the touchscreen feature of the iPad. From my observations and student comments, 
the ability to visualize a molecular structure in 3-D seems to really help students understand 
spatially-related concepts. 

When I joined our SoTL faculty learning community, I needed to identify a teaching- 
related topic that would be suitable for a research project. I delved into the literature and found 
many articles on spatial reasoning in chemistry and some articles on the use of iPads in 
chemistry (Amick & Cross, 2014; Harle & Towns, 2011; Hesser & Schwartz, 2013; Hofstein et 
al., 2013; McCollum, Regier, Leong, Simpson, & Sterner, 2014; Stieff, Ryu, Dixon, & Hegarty, 
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2012). I was excited about the fact that only a few articles had been published combining the two 
ideas and none of these publications discussed the applications or type of exercises I had 
developed. I was confident that a small research study would prove how valuable my approach 
was in improving students’ understanding of chemistry and, as a bonus, yield readily-publishable 
results. 

After conferring with my SoTL colleagues on the design of the research study, I decided 
to have my organic chemistry students answer a set of questions about several chemical 
structures after viewing them on the iPad in 3-dimensions (Chem3D app) and in 2-dimensions 
(ChemDraw app). My class was divided so that half of the students started the exercise in 2-D 
followed by 3-D and vice-versa. Presumably this would eliminate bias due to the “gadget factor” 
and really focus on the ability to “visualize the chemistry.” I was pretty sure I had a well- 
designed study that would clearly show how the iPad improved students’ spatial reasoning and 
ability to correctly answer challenging and important organic chemistry questions! 

In order to have sufficient time for students to complete this exercise, I scheduled it 
during our 3-hour lab period. The iPads and instructions were distributed and I reminded 
students to follow the order of questions as assigned and to work independently. Pretty quickly, I 
could see that students were not correctly answering several of the questions. Some of the 
weaker students were even struggling to understand the questions. I became concerned that this 
exercise was not going to end well. 

Then ... one of my students asked a question that confirmed my suspicions. I asked the 
students to assign the absolute configuration of chiral carbons in several of the structures. This is 
a typical quiz question and student’s ability to digitally-manipulate a chemical structure on the 
iPad should have made it really easy for them to answer the question. In addition, we had spent 
several weeks in the accompanying lecture course studying stereochemistry and the concept of 
chirality. Instead of progressing through the exercise as I had planned, this student, who was 
quizzically looking at the image of tartaric acid, said “What is a chiral carbon, anyway?” I 
wanted to say, “You’ve got to be joking,” but I didn’t. 

I realized that my expectations for many of the students had been too high. Instead, I 
needed to review many of the concepts I asked my students to use for this exercise. So, I 
abandoned any hope of obtaining publishable research data; I recognized that I needed to “re¬ 
teach” students and took the opportunity to do so by abandoning my beautifully-designed 
research experiment. 

So what did I learn? Most importantly, I recognized that I needed to scrap the research 
study and focus on helping my weaker students really understand the fundamentals of the topics 
I had been covering. I also learned that the exercise was probably “over-engineered.” The idea 
of visualizing an image in 2-D followed by 3-D (or vice versa) seemed to be cumbersome and 
confusing. For students who were already struggling with the material, this was an exercise in 
added frustration. 

In retrospect, I would have designed the study with easier questions and compared 
student answers with and without the use of an iPad. A simpler scaffolding approach would have 
been smarter and certainly less confusing to my students. However, from the study I attempted, I 
also learned to quickly regroup and address student questions about topics I believed were well- 
understood but really weren’t. As is often the case with SoTL projects, the teaching and learning 
aspects (mine) were clear, even if the scholarship piece didn’t go as planned. 

Truth, Lies, and Murphy’s Law 
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Karen Brown’s background is in nursing, specifically, gerontological nursing education. Her 
research interests include the effects of inter generational sendee-learning on student outcomes, 
and the impact of Human Patient Simulation, using high-tech mannequins, on nursing students’ 
leadership skills and emotioned intelligence. 

My SoTL research study protocol incorporated a form of deception to determine if one 
student’s tardiness affected other classmates’ test scores. Using deception in human subject 
research by intentionally providing misleading communication regarding the purpose or 
procedures of the research requires a solid rationale (U.S. Department of Health and Human 
Services, 2011). According to the Belmont Report, “all cases of research involving incomplete 
disclosure, such research is justified only if it is clear that (1) incomplete disclosure is truly 
necessary to accomplish the goals of the research, (2) there are no undisclosed risks to subjects 
that are more than minimal, and (3) there is an adequate plan for debriefing subjects” (U.S. 
Department of Health and Human Services, 1979, “Part C: Applications,” para. 6). Therefore, 
most Internal Review Boards (IRB) are stringent in their approval of the use of deception in 
research using human subjects. 

To begin, per the IRB’s instruction to minimize risks to the subjects, I recruited a 
confederate who was not a class member. In nursing, class cohorts know each other well, so I 
planned to conduct the experiment during the second week of class so the group would think this 
was a new member to their cohort. According to the experiment’s protocol, individuals were 
randomly assigned into two groups; first, the control group moved to a separate classroom with 
an independent assignment. Meanwhile, the experimental group watched a video and the 
confederate entered late. After that, the control group watched the same video; however, no late 
entries occurred during the control group’s video lesson. Finally, a post-test administered 
immediately upon conclusion of the video lesson assessed each group’s understanding of the 
lesson’s content and provided data needed to answer my research question. After the experiment, 
I reunited the class. At this time, I would debrief the class about the experiment, disclose the 
deception, provide informed consent allowing the use the post-test scores as part of the research, 
and note that the scores would not contribute to the course grade. 

Despite my meticulous study protocol, Murphy’s Law is alive and well, especially when 
it comes to SoTL research. As in my case, unexpected results can occur due to the complex 
interplay of course content, people, and processes inherent to SoTL research (Yeo, 2009). When 
my experiment went awry due to unexpected events outside of my control, I felt disenchanted. 
The first road block arose due to technology issues during the experimental group’s lesson. 
Although my confederate entered the room at the pre-arranged time, the lesson’s video did not 
play correctly; as a result the tardy confederate entered before the lesson began. Resiliently, I re¬ 
attempted the experiment during the next class session; this time, my confederate’s house 
flooded and she could not attend. I felt stressed and ready to give up. 

Fortunately, I was part of a FLC dedicated to SoTL research. The trusting bonds 
developed among the FLC allowed me to confess my experiment’s failure and the events leading 
to it. As a result, the group helped me to bolster my confidence, offered advice to alleviate 
technical issues, and encouraged me to request IRB approval to recruit a confederate from the 
class. Fortunately, the IRB approved my request, I restructured my experimental protocol (Yeo, 
2009) and, later during the same semester and course, successfully conducted the revised 
experiment. Ultimately, I presented the results of my SoTL project at a peer-reviewed, national 
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conference on teaching and learning. In short, the support given by my FLC prompted me to 
rebound, meet my teaching and learning goals, and beat Murphy’s Law. 

Compounding the unexpected challenges, I felt uncomfortable deceiving my students, 
even to acquire evidence of best teaching practices. I contemplated the lack of virtue of people 
who deceive others and the importance of role-modeling honesty in educational settings (Carr, 
2014). Importantly, I wondered if this form of deceit would erode the trust I was building with 
the students (Lount, Zhong, Sivanathan, & Murnighan, 2008). Therefore, at the conclusion of the 
successful experiment, I found myself reluctant to confess my deception to my students. 
Surprisingly, the students seemed astonished and excited to be part of an experiment; no one 
expressed dismay that deception was involved. Later, when I acknowledged my conflicted 
feelings about the deception to my FLC, the group validated my feelings while noting that 
deception may be appropriate to expand SoTL knowledge. Through my relationships with FLC 
members, I tapped into the supportive, trusting social network of faculty colleagues to gain 
courage, face failure, recover, and eventually flourish (Southwick & Charney, 2012). 

The ‘Accidental’ SoTL Project 

Theresa Conover is an assistant professor of criminal justice in the Department of Justice and 
Community Studies at Miami University. She teaches research methods, capstone, crime 
prevention, and policing courses. Conovers’s research interests include problem-solving 
approaches to address crime and quality of life issues, the criminology of place, and the role 
community engagement plays in crime prevention and public safety. 

One of the goals of our FLC was to design and implement a SoTL project. The group 
met during the spring semester with the expectation that our project would be carried out during 
the upcoming fall semester. The timing posed a challenge because my research leave was 
scheduled for the fall and I would not be on campus. Therefore, I needed to create an activity or 
project in one of the courses I was currently teaching in the spring. I had discussed a class 
project with the local police department to integrate into the Capstone 1 in Criminal Justice 
course. The police department was interested in gathering feedback from local businesses in the 
central business district to determine their needs, levels of satisfaction, and other concerns before 
adding a new dedicated officer to the area. This experiential learning project would create 
opportunities for students to examine the literature related to police-citizen satisfaction, identify 
the best way to gather the information, determine the methodology, collect and analyze data, and 
to present findings and recommendations (Conover, 2015a). 

Meanwhile, the FLC continued to meet, and I still had no research project. My failure 
was my inability to identify a project, but after discussing this with my FLC, it dawned on me: I 
did have a project and it was the experiential learning component of the capstone! The SoTL 
project would evaluate the student learning that took place using an existing pedagogy applied to 
a new discipline. In the process of searching for a SoTL project, I found my way to the literature 
that described a type of collaboration that occurs between universities, community partners, and 
students. It turns out that what I was doing had a name - it was community-based research 
(CBR). I found many examples of this pedagogy in other disciplines, such as teacher education, 


1 The Capstone course is designed to integrate and draw upon students’ previous learning in the program and 
provides the opportunity to apply the knowledge and skills they have accumulated throughout their undergraduate 
career. 
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sociology, and psychology; however, I did not find any examples of its use in criminal justice. 
CBR is comprised of three pillars: (1) collaboration between community, faculty, and students; 
(2) situation of research within an educational context; and (3) the purpose of addressing unmet 
needs in the community (Paul, 2006). All of these components were designed into the 
experiential learning portion of my course from the start. My SoTL project would assess the 
learning outcomes for the Capstone class. However, it was my engagement with other FLC 
members that helped me recognize that I already had most of my research project set up. 

Ideally, the assessment would use a pre-post design to compare the knowledge of 
research methods and skills before and after the course. However, the class had already started, 
and I had not done the pre-test. Fortunately, a member of the FLC suggested the use of the post- 
then-pre (retrospective post-then-pre) design which would accommodate this type of situation 
(Howard, 1980). Pre- and post-data are collected at the same time using the same instrument 
where questions are framed, for example, by asking “Before taking this course...” and “After 
taking this course...” in which students would respond to items measuring their perception of 
learning. 

My SoTL research project focused on the “what is” of SoTL (Hutchings, 2000) and 
detailed the CBR pedagogy applied to a new discipline and measured student learning (Conover, 
2015b). The student survey was administered to measure perceptions of knowledge and 
understanding of the research process, acquisition of research skills, and ‘softer’ skills which 
dealt with collaboration and professionalism. Results indicated statistically significant 
improvements in the perception of knowledge and understanding of the research process and 
survey methods and strong agreement that students had gained valuable professional skills while 
participating in the CBR project. 

It turns out that the SoTL research question that I struggled so much to identify was right 
in front of me all along. Ultimately, it involved something larger than a single activity; it 
involved a method of teaching students survey methodology through a hands-on approach to 
designing and guiding research to address a ‘real world’ issue in the community. In the end, I 
applied a teaching pedagogy that was new to my discipline. It was a new approach to teaching in 
my field and through the ‘what is’ narrative, provides an example from which others can learn 
and implement. 

Approaching Initial Findings as a ‘First Cut’ at the Research Question 

John Forren is an assistant professor in the Department of Justice and Community Studies at 
Miami University. A political scientist by training, Forren’s research and teaching interests 
focus on American politics, public law, criminal justice and civic engagement. The SoTL project 
described below grew out of the author’s work as the academic director of a U.S. State 
Department-sponsored experiential learning program on civic engagement held at Miami 
University in 2013, 2014 & 2015. 

My foray into SoTL research began with a rather straightforward empirical question: 
was an intensive summer program on civic engagement, specially designed to encourage active 
citizenship, actually producing the shifts in participants’ political attitudes that my colleagues 
and I were expecting it to yield? In the spring of 2014, when this SoTL project began, I had a 
practical reason to seek an answer to this question. An interdisciplinary team at Miami 
University was working at the time to plan our second Study of the U.S. Institute (SUSI) on 
Civic Engagement - a U.S. State Department-funded residential program that immersed student 
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leaders from Africa, the Middle East and the United States in learning about democratic theory, 
U.S. politics and techniques of community engagement. For the summer 2014 iteration of SUSI, 
we had carefully planned a curriculum that blended roughly 30 hours of formal classroom 
instruction with an array of site visits, group discussions, ‘hands-on’ civic skills workshops and 
other intensive experiential learning opportunities. Looking forward to an anticipated 2015 
version as well - and also viewing the 2014 SUSI as an ideal opportunity to ‘field test’ some 
hypotheses about the attitudinal effects of immersive civic engagement programs — we set out to 
leam: was our program meeting its aim of shaping students’ views about democracy, 
government and citizenship? 

Drawing from the relevant literatures in political science, college student development 
and experiential learning (Conway, Amel, & Gerwein, 2009; Hurtado, 2009; Zuniga, Williams, 

& Berger, 2005), I hypothesized that active participation in the SUSI program would lead 
participants toward (1) higher levels of personal support for freedom of thought and expression; 

(2) greater personal support for government activism in support of social and political equality; 

(3) broader understandings of the obligations associated with democratic citizenship; and (4) 
changed perceptions - perhaps in a negative direction - of how responsive the American political 
system is to ordinary citizens’ desires and concerns. To explore whether these attitudinal shifts 
were indeed taking place, I recruited volunteers from the 2014 cohort to complete identical 
paired 40-question surveys at the beginning and the end of their five-week SUSI experience. 
Using Likert scales to measure intensity of attitudes (Likert, 1932), these surveys elicited 
participants’ views on a range of contested ideological and political issues; each participant’s 
responses were then used to produce individual pre-SUSI and post-SUSI attitudinal scores along 
each of the four dimensions listed above (see McClosky, 1964). Paired T-tests were performed 
to determine whether the data supported our hypotheses regarding SUSI’s impact. 

The results, such as they were, fell generally in the expected direction. According to the 
data, at least some of the SUSI participants had indeed moved in the hypothesized directions for 
each of the measured variables. Sixty percent registered positive gains in their levels of support 
for freedom of expression and thought, a core democratic value that was discussed extensively in 
SUSI. Thirty percent of participants showed greater levels of support after SUSI for government 
action aimed at promoting equality. Half of the subject group revealed a broader understanding 
of the duties of citizenship at the end of their SUSI experience. At the same time, sixty percent 
registered increased skepticism about the degree to which U.S. politics fairly represents the 
views and needs of all of the nation’s citizenry. 

Such findings were certainly in line with our general expectations; what’s more, they 
provided at least some modicum of support for the idea that our SUSI program was 
accomplishing its central attitudinal goals. Yet it had also become clear by the end of the process 
that the 2014 study had fallen far short of providing definitive answers to the practical and 
theoretical questions that had motivated it. Two related weaknesses in the data - an unexpectedly 
small sample size and a correspondingly weak statistical relationship between program 
participation and attitudinal change - made it impossible to generalize from the data standing 
alone in a scientifically rigorous way. (Despite my best recruitment efforts, only 10 of the 
program’s 23 participants agreed to participate in the study.) Beyond that, a third unanticipated 
shortcoming in the study - the finding that many of our SUSI study participants had already 
possessed especially strong ‘pro-democracy’ attitudes even before starting the program - further 
weakened the vitality and generalizability of the results. 
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At the same time, two other considerations cautioned against our ‘throwing in the towel’ 
altogether on this research idea. For one, other sources of information that we had gathered on 
the impact of SUSI, including reflective narratives provided by participants throughout the five- 
week experience, suggested quite strongly that attitudes about democratic norms and citizenship 
had indeed changed in ways that our pre/post survey instrument perhaps had not captured. 
Second, by definition, the pre- and post-SUSI surveys addressed only the degree of attitudinal 
change that may have occurred over the five-week SUSI program itself- not any changes that 
may have occurred in later periods as the participants continued to absorb and reflect upon the 
experience. In this particular context, one could reasonably expect that a significant evolution in 
participants’ views about politics and democracy may still be underway even years later. 

So what to do? Sharing my concerns at one of our FLC meetings, several colleagues 
encouraged me to approach my results from a different perspective. Rather than simply 
abandoning the 2014 study as a failed experiment, they suggested, I might build upon its findings 
by gathering additional data which, when added to the 2014 results, could still yield significant 
conclusions that add meaningfully to the literatures on experiential learning and civic 
engagement. Taking that approach, I have since continued to track the 2014 SUSI cohort, hoping 
to discover whether evidence may still be found of long-term attitudinal changes within that 
group. Beyond that, I have also expanded my dataset by repeating the pre/post survey with 
another SUSI cohort in summer 2015 — an effort that, the second time around, thankfully yielded 
a higher proportion of study volunteers from among the program’s participants. With this larger 
set of survey data now in hand — and with a final round of interviews and surveys of the 2014 
and 2015 cohorts currently planned for summer 2017 — I now anticipate that my initial pre/post 
survey will indeed contribute, albeit in a way quite different than originally designed, to a 
publishable analysis of program impact. Looking back at it now, my initial 2014 study was 
clearly not a ‘failure,’ even though it indeed fell short of producing the generalizable results that 
I had initially anticipated. Rather, as my FLC colleagues helped me to see, the 2014 analysis is 
perhaps best understood as a ‘first cut’ at the questions it was aimed at exploring — and as such, 
an essential foundation for success in the broader project that it has produced. 

Conclusion 

As these research vignettes show, failures in research need not mark the discouraging end of 
once-promising ideas; rather, they can teach us important lessons, give us new insights and lead 
us down new paths of productive scholarly work. For the authors of this article, this realization — 
that lemons could indeed produce lemon trees — was the product of a sustained partnership with 
colleagues from a variety of disciplinary backgrounds and perspectives. What is more, it 
emerged from a FLC remarkable for its collaborative spirit, its sense of shared purpose and its 
openness to the sharing of frank but supportive feedback. While this FLC cohort represented 
multiple disciplines, we believe that we were able to help one another grow from our challenges 
and “failures” by encouraging each other to think more deeply about alternative possibilities; 
shared experiences of “failure” conceptually and experientially functioned as a generative 
commonality for alternative research inquiry. This confirms aspects of Sagor (2000) and 
Bradbury and Reason’s (2003) principles from their action research theories that highlight the 
collaborative results of such research as improving the skills and knowledge(s) of the people 
involved. 
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One main result of this interdisciplinary reflective action research was how this FLC 
group helped each other consider the appropriate and alternative use of research methods outside 
of one’s own discipline, even if this insight was gained in hindsight. For example, Kuznekoff and 
Forren employed quantitative research method approaches that could have been complimented 
by a better understanding of their respective student populations prior to the study. Kuznekoff, 
specifically, made assumptions about his student population and students note-taking skill level. 
Perhaps a quick assessment of students’ note-taking abilities prior to the experiment would have 
created a baseline for the author to work from or, at the very least, help to support his 
explanation for the lack of statistically significant results. Similarly, Forren, in designing his 
initial study, did not sufficiently account for the effect that self-selection would have on the 
population of students in the program being studied. In both of these instances, we concluded 
that alternative research perspectives from other disciplines could have been helpful for 
designing such research and unpacking initial assumptions can help prevent design flaws and 
failures. 

Other narratives in this article, such as Dich and Conover’s account speak to the ways 
that FLC’s can help researchers maintain an open and flexible approach to difficult or 
unexpected research circumstances. As Dich noted, she did not receive any participation consent 
in her study and initially considered her study a failure. Yet, her reflection demonstrated the need 
to ask different questions of a seemingly simple project. In fact, it was in failing that allowed her 
to consider a completely different set of inquiry, one that could be helpful to other researchers of 
online teaching. Conover’s account also presented difficult research circumstances with work 
and time limitations, but such difficulty also forced Conover to reconsider an alternative research 
direction and a new pedagogical approach for her own teaching and field: “it involved a method 
of teaching students survey methodology through a hands-on approach to designing and guiding 
research to address a ‘real world’ issue in the community.” In this way, Conover was able to 
innovate both research and teaching for her own discipline, an important outcome that can be 
overlooked with SoTL FLCs. 

What we also see from Brown and Marshall is a narrative of ethical inquiry in relation to 
“failure.” Theoretically, SoTL researchers understand that an ethical approach to research means 
putting the best interest of the students first. Yet, this theory doesn’t always translate well to 
practice. This is not to say that researchers are not ethical, but that the benefit of being in an FLC 
is being able to assess and discuss ethical issues within an interdisciplinary group setting. As 
Brown showed, deceiving her students is a struggle that may emerge from the discipline-specific 
field of nursing, where ethics and trust are much more engaged within the teaching and practice 
of nursing. Yet, such ethical struggles may not be something other disciplines will see as an 
issue. Although Brown’s students did not express an issue with being deceived, it did help other 
members of the FLC consider ethics within their own field, as evidenced by Marshall's narrative. 
Marshall demonstrated a moment of practiced ethics in which she made the decision to end her 
research in favor of student learning. This particular narrative reminded the group that the 
fundamental goal of FLC’s and its research is to help teachers teach better and students improve 
learning outcomes. In the midst of research design and implementation, it is easy to forget the 
reason for such research. What this narrative can teach us is how an “over-engineered” 
experiment can set both the project and students up for failure and frustrations, regardless of 
discipline. But our first responsibility is to the students, and this can mean “scrapping” an 
experiment in favor of learning outcomes. 
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We also want to point out the institutional support that helped create the context for our 
continued inquiry and action research. We wanted to emphasize that such FLC’s on campuses 
are beneficial to the whole campus because faculty can contribute their learning to their 
respective departments/disciplines, which can diminish some of the “silo” effect within 
academia. Ultimately, breaking down of silo-like knowledge can help bring different 
departments and areas of knowledge together to better serve students and enhance educational 
purposes. For instance, through the FLC, Kuznekoff and Marshall were able to collaborate on 
ways that iPads can be used in the classroom. While Kuznekoff was interested in notetaking 
outcomes, such a focus helped Marshall consider the use of iPads as a research and learning tool 
in chemistry labs. The knowledge gained from both of these studies goes beyond their respective 
disciplines and can be just as useful in other disciplines, but this sharing of knowledge would be 
unlikely without the FLC structure. Administrators or faculty considering FLC models should 
also consider the value of enlisting a diverse group of faculty who can represent multiple 
disciplines: members can contribute as experts in their own field, but grow from the engagement 
with other disciplines and perspectives. For example, our collective experience indicates that 
FLCs may do well with an initial activity asking members to unpack and examine personal and 
disciplinary assumptions about our students and each other. Such self-reflexivity may help us see 
the weaknesses in our designs, expose us to other disciplinary methods and tools, and avail us to 
other forms of knowledge-making. 
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